Fully-connected Neural Network Research Articles

A bayesian deep learning-based approach to group contribution methods: A study on cannabinoids and terpenes

Although there are several group contribution methods available for predicting thermodynamic properties, the uncertainties associated with these predictions are unknown. In this study, we present a new group contribution method approach based on bayesian neural networks (BNNs) in order to provide predictions and their associated uncertainty. Various machine learning techniques, including a fully connected neural network (FCNN) and a graph neural network (GNN) under the BNNs framework, were employed to efficiently implement this methodology. As a case of study, we focused on predicting the melting and boiling points of cannabinoids and terpenes, as well as their corresponding uncertainties. Individual analyses were carried out for boiling and melting points using databases containing 2529 and 2862 chemical compounds, respectively. Additionally, we conducted an assemble study of both properties using a database of 1503 chemical compounds. To validate the models, a database consisting of 47 cannabinoids and terpenes was employed. The models exhibited exceptional prediction results for boiling points, showcasing coefficients of determination R2≥ 0.9 for all the studied models. On the contrary, only the assemble GNN model yielded accurate melting point predictions with an R2 value of 0.94, while the R2 values obtained from the other models ranged from 0.51 to 0.66. Finally, our predictions for the melting and boiling points of two well-known cannabinoids, CBD and THC, are (70.5±45.4) °C and (419.4±25.4) °C for CBD, and (87.0±46.7) °C and (411.8±29.7) °C for THC, respectively.

Crash Prediction on Horizontal Curves: Review and Model Performance Comparison

More than 25% of all roadway fatalities in the U.S. are associated with a horizontal curve, and the average crash rate for horizontal curves is about three times that of other types of highway segments. A focus on horizontal curves can prove to be a cost-effective approach to reducing safety issues. Accurate crash prediction models (CPMs) on horizontal curves can help roadway safety practitioners assess and prioritize safety improvements. Although many CPMs have been developed, there are no extant studies that compare different CPMs on a singular, real-world, large-scale, and comprehensive dataset to evaluate their capability for horizontal curve crash prediction. This study critically evaluated commonly used CPMs, including multiple linear regression (MLR), Poisson regression (PR), negative binomial regression (NBR), support vector machine, random forest (RF), and fully connected neural network (FCNN) models, on rural curves extracted from 18,000 centerline miles of Georgia, U.S.’s state-maintained routes and statewide historical crash data set from 2013 to 2021. Results show PR and NBR models outperform MLR by around 6%. Moreover, the FCNN and RF models further improved this performance by around an additional 6% over the PR and NBR models. Overall, machine learning (ML)-based models outperform generalized linear regression models. The results prove ML-based models can be recommended to transportation agencies to forecast horizontal curve crashes more accurately.

Electromagnetic interference shielding composite aerogels with asymmetric structures developed in aid of neural network

Along with the increasingly serious electromagnetic interference (EMI) pollution, it is in urgent need of exploiting high-performance EMI shielding materials. However, the conventional experimental methods always require a series of tedious preparation and characterization processes, resulting in the long period and high cost. Herein, a fully connected neural network (FCNN) was applied to aid in exploiting silver nanowires/waterborne polyurethan/multi-walled carbon nanotubes (AgNWs/WPU/MWCNTs) EMI shielding aerogels with asymmetric structures to optimize new material evolution and reduce experiment burden. Firstly, the special AgNWs/WPU/MWCNTs aerogels were fabricated with various MWCNTs content (0 wt.%, 2 wt.%, and 4 wt.%) and AgNWs mass (10 mg, 20 mg, 30 mg, and 40 mg) via a combination of directional freezing and spraying procedures, leading to 12 distinct samples. AgNWs/WPU/MWCNTs aerogels hold excellent EMI shielding properties and absorption-dominant shielding mechanism. The experimental EMI shielding effectiveness (SE) data at different frequency were leveraged to establish, train, and test the FCNN model based on Adam optimizer (Adam-FCNN). It is worth noting that Adam-FCNN model predictions are in good accordance with the EMI SE of the physically measured samples, with the lowest root mean square error RMSE of 0.7899, the highest correlation coefficient (R) of 0.9895, and excellent computational efficiency and reliability. Moreover, Adam-FCNN can accurately prognosticate EMI SE of unobtained AgNWs/WPU/MWCNTs aerogels in mechanics-intuitions to reduce repeated preparation and characterizing conditions. Thereby, Adam-FCNN soft model approach opens significant potential for alleviating the experimental burden, accelerating the discovery of innovative materials, and exploring the complex mechanism.

Integrating CEDGAN and FCNN for Enhanced Evaluation and Prediction of Plant Growth Environments in Urban Green Spaces

Conducting precise evaluations and predictions of the environmental conditions for plant growth in green spaces is crucial for ensuring their health and sustainability. Yet, assessing the health of urban greenery and the plant growth environment represents a significant and complex challenge within the fields of urban planning and environmental management. This complexity arises from two main challenges: the limitations in acquiring high-density, high-precision data, and the difficulties traditional methods face in capturing and modeling the complex nonlinear relationships between environmental factors and plant growth. In light of the superior spatial interpolation capabilities of CEDGAN (conditional encoder–decoder generative adversarial neural network), notwithstanding its comparative lack of robustness across different subjects, and the excellent ability of FCNN (fully connected neural network) to fit multiple nonlinear equation models, we have developed two models based on these network structures. One model performs high-precision spatial attribute interpolation for urban green spaces, and the other predicts and evaluates the environmental conditions for plant growth within these areas. Our research has demonstrated that, following training with various samples, the CEDGAN network exhibits satisfactory performance in interpolating soil pH values, with an average pixel error below 0.03. This accuracy in predicting both spatial distribution and feature aspects improves with the increase in sample size and the number of controlled sampling points, offering an advanced method for high-precision spatial attribute interpolation in the planning and routine management of urban green spaces. Similarly, FCNN has shown commendable performance in predicting and evaluating plant growth environments, with prediction errors generally less than 0.1. Comparing different network structures, models with fewer hidden layers and nodes yielded superior training outcomes.

Training an Ising machine with equilibrium propagation

Ising machines, which are hardware implementations of the Ising model of coupled spins, have been influential in the development of unsupervised learning algorithms at the origins of Artificial Intelligence (AI). However, their application to AI has been limited due to the complexities in matching supervised training methods with Ising machine physics, even though these methods are essential for achieving high accuracy. In this study, we demonstrate an efficient approach to train Ising machines in a supervised way through the Equilibrium Propagation algorithm, achieving comparable results to software-based implementations. We employ the quantum annealing procedure of the D-Wave Ising machine to train a fully-connected neural network on the MNIST dataset. Furthermore, we demonstrate that the machine’s connectivity supports convolution operations, enabling the training of a compact convolutional network with minimal spins per neuron. Our findings establish Ising machines as a promising trainable hardware platform for AI, with the potential to enhance machine learning applications.

DeFiTrust: A transformer-based framework for scam DeFi token detection using event logs and sentiment analysis

Decentralized finance (DeFi) token scams are rising, leading to significant losses for investors and users. Hence, it is vital to identify malicious DeFi tokens accurately and early to protect investor funds. Current Machine Learning models that detect scam tokens rely entirely on DeFi token transactional behavior and have overlooked the public opinion that can be derived from news and social media posts, which can provide helpful information for identifying scam tokens. Further, existing frameworks to detect scam tokens fail to explain why certain tokens are identified as scams, hindering stakeholder confidence in such systems. This paper proposes a new transformer-based framework called “DeFiTrust” to identify malicious DeFi tokens using transaction events and social media posts. DeFiTrust has two components: event log processing and sentiment analysis. The former creates a feature vector representing temporal variations in the event logs of a token. The latter analyzes social media posts to determine public opinion about the token. A fully-connected neural network then processes both pieces of information to determine whether the token is legitimate or a scam. We evaluated the proposed model against existing Machine Learning-based models using real-world data. The results demonstrate that DeFiTrust outperforms existing approaches and detects scam tokens earlier. Explanations generated using Integrated Gradients reveal that DeFiTrust identifies scam tokens based on valid reasons, enhancing stakeholder trust in the proposed model.

Off-axis telescope misalignment correction based on defocus spot moment features

Optical mirror misalignments, which are caused by assembly mistakes and changes in the surrounding environment (such as gravity, temperature, and atmosphere), degrade the system’s imaging performance. Therefore, active misalignment correction is essential for ensuring the image quality of the off-axis telescope. In this paper, a novel misalignment correction method without wavefront sensors is proposed. The point spread functions (PSFs) of the system are analytically related to the optical mirror misalignments. On this basis, a fully connected neural network (FCNN) is used to establish the mapping relationship between the misalignments and the discrete orthogonal unbiased finite impulse response (UFIR) moment features, which can effectively characterize changes of intensity and geometric of the spot image. The simulation and experimental results in this paper justify the effectiveness and practicality of the proposed method. This approach offers a low-cost and straightforward technical method for achieving high imaging quality throughout the alignment and observation phases. This approach can prevent the accumulation of errors caused by wavefront detection and the high delay of multiple iterations.

A subject-specific unsupervised deep learning method for quantitative susceptibility mapping using implicit neural representation

Quantitative susceptibility mapping (QSM) is an MRI-based technique that estimates the underlying tissue magnetic susceptibility based on phase signal. Deep learning (DL)-based methods have shown promise in handling the challenging ill-posed inverse problem for QSM reconstruction. However, they require extensive paired training data that are typically unavailable and suffer from generalization problems. Recent model-incorporated DL approaches also overlook the non-local effect of the tissue phase in applying the source-to-field forward model due to patch-based training constraint, resulting in a discrepancy between the prediction and measurement and subsequently suboptimal QSM reconstruction. This study proposes an unsupervised and subject-specific DL method for QSM reconstruction based on implicit neural representation (INR), referred to as INR-QSM. INR has emerged as a powerful framework for learning a high-quality continuous representation of the signal (image) by exploiting its internal information without training labels. In INR-QSM, the desired susceptibility map is represented as a continuous function of the spatial coordinates, parameterized by a fully-connected neural network. The weights are learned by minimizing a loss function that includes a data fidelity term incorporated by the physical model and regularization terms. Additionally, a novel phase compensation strategy is proposed for the first time to account for the non-local effect of tissue phase in data consistency calculation to make the physical model more accurate. Our experiments show that INR-QSM outperforms traditional established QSM reconstruction methods and the compared unsupervised DL method both qualitatively and quantitatively, and is competitive against supervised DL methods under data perturbations.

Large eddy simulation of flow over a circular cylinder with a neural-network-based subgrid-scale model

A neural-network-based large eddy simulation is performed for flow over a circular cylinder. To predict the subgrid-scale (SGS) stresses, we train two fully connected neural network (FCNN) architectures with and without fusing information from two separate single-frame networks (FU and nFU, respectively), where the input variable is either the strain rate (SR) or the velocity gradient (VG). As the input variables, only the grid-filtered variables are considered for the SGS models of G-SR and G-VG, and both the grid- and test-filtered variables are considered for the SGS models of T-SR and T-VG. The training data are the filtered direct numerical simulation (fDNS) data at $Re_d=3900$ based on the free-stream velocity and cylinder diameter. Using the same grid resolution as that of the training data, the performances of G-SR and G-VG (grid-filtered inputs) and T-SR-FU and T-VG-FU (grid- and test-filtered inputs with fusion) are better than those of the dynamic Smagorinsky model and T-SR-nFU and T-VG-nFU (grid- and test-filtered inputs without fusion). These FCNN-based SGS models are applied to untrained flows having different grid resolutions from that of training data. Although the performances of G-SR and G-VG are degraded, T-SR-FU and T-VG-FU still provide good performances. Finally, T-SR-FU and T-VG-FU trained at $Re_d = 3900$ are applied to higher-Reynolds-number flows ( $Re_d = 5000$ and 10 000) and their results are also in good agreements with those of fDNS and previous experiment, indicating that adding the test-filtered variables and fusion increases the prediction capability even for untrained Reynolds number flows.

Stock price forecasting by supervised learning methods

Predicting stock prices has already long been a popular but challenging topic due to unavoidable uncertainties in various factors. With continuous exploration and development of machine learning algorithms over the years, they have become more and more popular in forecasting stock value changes. This paper aiming to predict future stock prices using returns from the past by applying three different supervised learning methods to the seemingly unpredictable dataset, namely Long Short-Term Memory (LSTM), Linear regression and Fully connected Neural Networks (FNN). R-squared and MSE were used as statistical indicators of the performance of each model. The calculated R-squared values for linear regression, LSTM and FNN were 0.876975, 0.898741 and 0.929504 respectively. In addition, the MSE of the corresponding models were 5.7786893, 0.0007542 and 0.0005237. As a conclusion, FNN performed the best in the sense that it put out the highest R-squared and lowest MSE value. As a result, FNN may take priority to the other manners in future predictions.

An ultra-low-computation model for understanding sign languages

In artificial intelligence applications, advanced computational models, such as deep learning, are employed to achieve high accuracy, often requiring the execution of numerous operations. Conversely, lightweight computational models are typically more resource-efficient, making them suitable for various devices, including smartphones, tablets, and wearable technology. This paper presents an ultra-low-computation solution for interpreting sign languages to assist deaf and hard-of-hearing individuals without needing specialized hardware or significant computational resources. The proposed approach initially performs data abstraction on the input data. During this process, the image is systematically scanned from various perspectives, and the collected information is then encoded into a one-dimensional vector. Subsequently, the abstracted information undergoes processing through a Fully Connected Neural Network (FCN), resulting in highly accurate output. We also introduced two abstraction methods, namely Opaque and Glass, inspired by the interaction of light with different types of objects. The proposed abstractions facilitate the comprehension of the hand gesture’s outer boundary as well as its row-wise and column-wise density of pixels. Our experiments on three datasets confirm the efficiency of the proposed method, achieving an accuracy of 99.4% in recognizing American Sign Language, 99.96% accuracy in recognizing Indian Sign Language, and 99.95% accuracy in recognizing Bangla Sign Language. Notably, the model size and the number of MAC operations are significantly smaller than state-of-the-art computational models trained on the same datasets.

Probabilistic state estimation in district heating grids using deep neural network

Probabilistic state estimation is critical for operating and controlling district heating grids efficiently. However, computational bottlenecks of traditional solvers limit the feasibility of uncertainty-aware Bayesian estimation. This paper proposes using deep neural networks (DNNs) to enable fast and accurate posterior estimation. Fully-connected neural networks (FCNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs) are evaluated as candidate approximators of the physical model. Markov chain Monte Carlo sampling in the heat exchange space is leveraged to generate posterior samples. Experiments on a benchmark heating grid demonstrate FCNNs can efficiently learn the mapping from heat exchanges to network states. A FCNN trained on 20 training epochs after hyperparameter optimization provides the best approximation accuracy and uncertainty estimates, outperforming prior methods based on Deep Neural Networks. The results highlight the potential of data-driven deep learning models for probabilistic state estimation. The proposed framework could enable real-time uncertainty-aware control and decision-making for future intelligent district heating grids.

Improving short-term offshore wind speed forecast accuracy using a VMD-PE-FCGRU hybrid model

The integration of large-scale offshore wind power into the power grid presents significant challenges for grid operation and dispatch due to the variability and intermittency of offshore wind energy. However, non-stationarities and noise in wind speed time series pose challenges. This study proposes a VMD-PE-FCGRU methodology to address this challenge of predicting offshore wind speeds. Firstly, the Variational Mode Decomposition (VMD) algorithm decomposes the original wind speed signal, considering the strong fluctuations and high noise inherent to offshore wind energy. A corresponding performance evaluation index is established to assess the effectiveness of this deconstruction process. To further enhance the model's (GRU) ability to capture time series information, a Position Encoding layer (PE) has been added, and the network is deepened through a Fully Connected Neural Network (FCNN). This allows multi-dimensional time series features to be input into the Gated Recurrent Unit (GRU) network for forecast. The accuracy of the proposed method is then verified using measured historical data from the wind tower of an offshore wind farm in Guangdong. Experimental results demonstrate that the proposed method can accurately forecast short-term wind speeds for offshore wind farms, with a Mean Absolute Error (MAE) of 0.199 and a Mean Absolute Percentage Error (MAPE) of only 2.45%. Moreover, the qualified rate (r) reaches 100%, providing an accurate reference for real-time power grid dispatching and can inform decision-making for the integration of offshore wind power into the grid.

Bisection Neural Network Toward Reconfigurable Hardware Implementation.

A hardware-friendly bisection neural network (BNN) topology is proposed in this work for approximately implementing massive pieces of complex functions in arbitrary on-chip configurations. Instead of the conventional reconfigurable fully connected neural network (FC-NN) circuit topology, the proposed hardware-friendly topology performs NN behaviors in a bisection structure, in which each neuron includes two constant synapse connections for both inputs and outputs. Compared with the FC-NN one, the reconfiguration of the BNN circuit topology eliminates the remarkable amount of dummy synapse connections in hardware. As the main target application, this work aims at building a general-purpose BNN circuit topology that offers a great amount of NN regressions. To achieve this target, we prove that the NN behaviors of the FC-NN circuit topologies can be migrated to the BNN circuit topologies equivalently. We introduce two approaches including the refining training algorithm and the inverted-pyramidal strategy to further reduce the number of neurons and synapses. Finally, we conduct the inaccuracy tolerance analysis to suggest the guideline for ultra-efficient hardware implementations. Compared with the state-of-the-art FC-NN circuit topology-based TrueNorth baseline, the proposed design can achieve 17.8- 22.2× hardware reduction and less than 1% inaccuracy.

Electrical performance of conductive cementitious composites under different curing regimes: Enhanced conduction by carbon fibers towards self-sensing function

Electrical performance is a significant indicator for conductive self-sensing cement-based composites, while the application of the composites is largely restricted in cold region due to the harsh environment. In this work, the electrical performance of carbon fibers reinforced conductive cement-based composites (CFR-CCBC) was evaluated, aiming to disclose the potential of ohmic heating (OH) cured conductive composites for structural health monitoring at cold region. Results showed that both OH curing and freezing condition were detrimental to the electrical conductivity, while the increase of CFs was an effective strategy to stabilize the electrical performance. However, the increasing CFs fraction was not economical and may induce strength degradation for concrete. At this time, the use of antifreeze could significantly stabilize the electrical conductivity by lowering CFs contents in CFR-CCBC with comparable electrical performance, even in severely cold environment. Piezoresistive property indicated that the combined use of 0.75 vol% CFs and antifreeze significantly increased gauge factor and linearity, indicating the improved self-sensing ability. Further, the fully connected neural network (FCNN) was employed to establish a model to systematically analysis the multiple factors on the electrical performance of CFR-CCBC.

Manufacturing process encoding through natural language processing for prediction of material properties

Knowledge of manufacturing processes is crucial to determine the final properties of a material, thus this work focuses on analyzing the relationship between final properties, chemical composition, and manufacturing process through data analysis. Furthermore, techniques of natural language processing are used to encode the manufacturing process as input in the neural network. The work consisted of two main parts: firstly, the relevant data was gathered, cleaned-up, and analyzed using statistical and probabilistic methods, K-means, and Principal Components Analysis (PCA), and secondly, a model was developed to predict elongation, yield, and tensile strength. Fully Connected Neural Network (FCNN) algorithms were used to build the aforementioned model. In addition, in order to avoid overfitting and evaluate the model dropout function and K-fold cross-validation are incorporated. Results demonstrated reasonable accuracy in elongation, yield, and tensile strength. Two cases of mechanical properties prediction of stainless steel alloy were presented, first, an existing alloy that was not in the training and test set, and second a suggestion of a new stainless steel alloy, which combines good YS, UTS, and Pitting resistance equivalent number (PREN). Additionally, it was considered an example from the TRIP family to showcase the tool’s versatility across various steel types.

Deep neural networks can differentiate thyroid pathologies on infrared hyperspectral images

Background and objectiveThe thyroid is a gland responsible for producing important body hormones. Several pathologies can affect this gland, such as thyroiditis, hypothyroidism, and thyroid cancer. The visual histological analysis of thyroid specimens is a valuable process that enables pathologists to detect diseases with high efficiency, providing the patient with a better prognosis. Existing computer vision systems developed to aid in the analysis of histological samples have limitations in distinguishing pathologies with similar characteristics or samples containing multiple diseases. To overcome this challenge, hyperspectral images are being studied to represent biological samples based on their molecular interaction with light. MethodsIn this study, we address the acquisition of infrared absorbance spectra from each voxel of histological specimens. This data is then used for the development of a multiclass fully-connected neural network model that discriminates spectral patterns, enabling the classification of voxels as healthy, cancerous, or goiter. ResultsThrough experiments using the k-fold cross-validation protocol, we obtained an average accuracy of 93.66 %, a sensitivity of 93.47 %, and a specificity of 96.93 %. Our results demonstrate the feasibility of using infrared hyperspectral imaging to characterize healthy tissue and thyroid pathologies using absorbance measurements. The proposed deep learning model has the potential to improve diagnostic efficiency and enhance patient outcomes.

APIB-GAN: A generative adversarial networks based approach for anomaly prediction of internet behavior

Anomaly detection focuses on finding non-conforming patterns of expected behavior in the data. In various application fields, these non-conforming patterns are often called anomalies, outliers, etc. With the wide spread of the Internet in various application fields, a large amount of Internet behavior log data is generated, which contains lots of private information. To protect sensitive data, the Internet behavior data of employees in a company are studied. Because of the high cost of manual labeling data, an unsupervised generative model, Anomaly Prediction of Internet behavior based on Generative Adversarial Networks (APIBGAN), is proposed to predict anomalies in online behaviors only with a small amount of labeled data. Three categories of Internet behavior data are studied with APIBGAN: (1) Online behavior data of an employee within a department. (2) Online behavior data of multiple employees in a department. (3) Online behavior data of multiple employees in multiple departments. The prediction accuracy of the above three cases is 87.23 %, 85.13 %, and 83.47 %, respectively, which shows that the outlier of Internet behavior can be accurately predicted only through a three-layers Fully Connected Neural Network (FNN) and it is more effective with the simple data distribution. The experimental results prove that Generative Adversarial Networks (GAN) has broad application prospects for anomaly prediction of Internet behavior data and those data with large volume that are not easy to manual labeling, such as financial data, biomedical data, etc.

Deep Learning for Wind and Solar Energy Forecasting in Hydrogen Production

This research delineates a pivotal advancement in the domain of sustainable energy systems, with a focused emphasis on the integration of renewable energy sources—predominantly wind and solar power—into the hydrogen production paradigm. At the core of this scientific endeavor is the formulation and implementation of a deep-learning-based framework for short-term localized weather forecasting, specifically designed to enhance the efficiency of hydrogen production derived from renewable energy sources. The study presents a comprehensive evaluation of the efficacy of fully connected neural networks (FCNs) and convolutional neural networks (CNNs) within the realm of deep learning, aimed at refining the accuracy of renewable energy forecasts. These methodologies have demonstrated remarkable proficiency in navigating the inherent complexities and variabilities associated with renewable energy systems, thereby significantly improving the reliability and precision of predictions pertaining to energy output. The cornerstone of this investigation is the deployment of an artificial intelligence (AI)-driven weather forecasting system, which meticulously analyzes data procured from 25 distinct weather monitoring stations across Latvia. This system is specifically tailored to deliver short-term (1 h ahead) forecasts, employing a comprehensive sensor fusion approach to accurately predicting wind and solar power outputs. A major finding of this research is the achievement of a mean squared error (MSE) of 1.36 in the forecasting model, underscoring the potential of this approach in optimizing renewable energy utilization for hydrogen production. Furthermore, the paper elucidates the construction of the forecasting model, revealing that the integration of sensor fusion significantly enhances the model’s predictive capabilities by leveraging data from multiple sources to generate a more accurate and robust forecast. The entire codebase developed during this research endeavor has been made available on an open access GIT server.

Data-driven approach for modeling Reynolds stress tensor with invariance preservation

The present study represents a data-driven turbulent model with Galilean invariance preservation based on machine learning algorithm. The fully connected neural network (FCNN) and tensor basis neural network (TBNN) (Ling et al., 2016) are involved and applied. The models are trained based on five kinds of flow cases with Reynolds Averaged Navier–Stokes (RANS) and high-fidelity data. The mappings between two invariant sets, mean strain rate tensor and mean rotation rate tensor as well as additional consideration of invariants of turbulent kinetic energy gradients, and the Reynolds stress anisotropy tensor are trained. The prediction of the Reynolds stress anisotropy tensor is treated as user’s defined RANS turbulent model with a modified turbulent kinetic energy transport equation. The results show that both FCNN and TBNN models can provide more accurate predictions of the anisotropy tensor and turbulent state in square duct flow and periodic flow cases compared to the RANS model. The machine learning based turbulent model with turbulent kinetic energy gradient related invariants can improve the prediction precision compared with only mean strain rate tensor and mean rotation rate tensor based models. The TBNN model is able to predict a better flow velocity profile compared with FCNN model due to a prior physical knowledge.